The present invention relates to a learning system for a neural network for pattern recognition, or more in particular to a learning pattern showing method for a neural network.
Technological fields in which the pattern recognition techniques are often used include voice recognition, character recognition, image recognition and drawing recognition. A conventionally-used method of statistical discrimination development of a method for calculating a feature value is required for discrimination. An improved discrimination rate based on superior discrimination of features like personal characteristics is necessary especially in voice and character recognitions. When the object of discrimination undergoes a change, however, it is not an easy matter to change the method of discrimination and the data for discrimination such as the feature dictionaries. A neural network, which utilizes a learning function, not only eliminates the need of development of a discrimination method but also makes it possible to equip the system with an adaptive discrimination ability.
The neural network uses the learning with a back propagation, as will be described. In a learning algorithm using a back propagation, which represents a supervised learning, it is necessary to prepare a value to be outputted by a neural network as a supervised pattern against an input pattern. An output value is produced through a neural network in response to an input pattern, and this output value is compared with the value of the supervised pattern corresponding to the input pattern. If the resulting error is larger than a reference, it is decided that convergence criteria are not satisfied and the weights of the neural network are updated to effect proper learning.
The learning using a back propagation poses the problems of a long calculation time for learning and local minima leading to a delayed learning. More specifically, the back propagation is based on an optimization method, or what is called the "mountain climbing" in the method of operations research, and therefore a right optimum point cannot be found unless the step size of the iterative calculation or moment value is set properly. In view of this, a "Method of Improving the Neural Network Learning Efficiency" disclosed in JP-A-01-320565 is intended to obviate the problems of pattern recognition including character and voice recognitions which occur at the time of learning of a neural network. This method consists in changing the step size and moment value in the process of iterative calculations of learning and thereby correctly proceeding with the iterative calculations of learning.
The method of proceeding with the learning depends on the method of selecting parameters such as step size and moment in the optimization on the one hand and the characteristics of the pattern for learning on the other hand. A method has been disclosed in which a pattern is divided into several groups in advance and the learning is effected separately for each group, followed by an overall learning to improve the speed. (JP-A-02-219167).
In the back propagation, a plurality of patterns are learned iteratively, and therefore different pattern learnings have different rates of progress. Generally, in the latter half of learning, a multiplicity of learning patterns have already gone through the learning in many cases. In view of this, a method is disclosed in which the weights are not corrected for patterns completed in learning thereby to shorten the calculation time as a whole. (JP-A-02-220169).
One cause of a long calculation time required and failure of correct calculation in the back propagation learning lies in the fact that it sometimes occurs that very similar patterns in a pattern set to be learned are classified into quite different categories, or totally different patterns are classified into the same category. The back propagation, which is called "the supervised learning", requires a correct supervised signal. The supervised signal is for determining which category a pattern belongs to and is ordinarily assigned by a human being, who may prepare a wrong supervised signal. If a supervised signal is erroneous, the learning calculation for back propagation is repeated more times than needed, and in some cases, indefinitely along a loop. Increased calculations occur when an input pattern does not contain a sufficient amount of information as well as when a supervised signal contains inconsistent information. In other words, a lack of information makes classification impossible. Due to this characteristic of back propagation, a method of excluding exceptional patterns in advance is disclosed by JP-A-02-235170. According to this method, only an effective pattern for learning is selected in advance by a statistical technique. An "effective pattern" is one belonging to the boundary between different patterns. A pattern located in the central part of a pattern is stated to protract the learning pattern or a pattern that has entered a category region lengthens unnecessarily the learning time thereby to reduce the classification performance for patterns other than a learning sample. Nevertheless, no consideration is made about a method of detecting and removing exceptional patterns during the learning.
In the case where back propagation is used as a learning method for a neural network, the learning pattern requiring discrimination is repeated and shown to the network a number of times to update the weights. In the conventional method in which a learning pattern is shown a number of times in similar fashion, the failure to end the learning calculation is posed as a problem if the supervised signal of the learning pattern is erroneous and contains an inconsistent pattern. Also, there may be a learning pattern which is not erroneous but very difficult to discriminate. In such a case, the learning calculation may end but require a very long length of time.